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I Since the need for good programers exists and will increase, their identification 

l before training is desirable. Until now only single tests of potential ability have been 

| evaluated. In this study several tests used in various combinations were evaluated as 

! test batteries. The IBM Aptitude Test for Programmer Personnel (ATPP) and the 

[ Strong Vocational Interest Blank (SVIB) were administered to 68 students enrolled in 

I an introductory computer science course at Michigan State University. Grade point 

i average (GPA) and College Qualification Test (COT) scores for participants were 

I available from college records. All test scores correlated significantly with course 

! grade (p .05). GPA was found to be the best single predictor or success. Among total 

| test scores, general scholastic aptitude (the CQT) predicted achievement as well as 
| specialized aptitude (the ATPP). However, the best results were found to be 

| obtainable with a judicious choice of subtests emphasizing numerical and spatial 

I reasoning (ATPP Part III, CQT Numerical). An “interest" variable as assessed by the 

I SVIB appeared to identify a dimension discrete from aptitude that was significantly 

I related to course achievement. The results were taken to indicate that instruments 

I presently available can be used effectively to predict achievement in computer 

I programing. (SS/MF) 
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PREDICTING PERFORMANCE IN A COMPUTER 
PROGRAMMING COURSE 

Roger Bauer 9 William A* Mehrens and John F. Vlnsonhaler 

Michigan State University 

Computer programming Is one of the fastest growing of the new technical 
professions. Large numbers of programmers are needed Immediately, and It Is 
predicted (Seller, 1965) that by 1970 there will be a half million new jobs 
In electronic data processing. Computers are more and more being forced to 
perform well below their potential for lack of programmers. 

Automatic data processing has had a significant Impact on private business 
schools; most of the larger ones now have computer-training programs for opera- 
tors and technicians (Fulton, In press). Too often, however, specialized 
schools accept students with scant aptitude for computer training; and un- 
qualified graduates ultimately heighten rather than alleviate the critical 
shortage of personnel In this area. Thus the need Is apparent for tests 
capable of Identifying potentially adequate computer programmers. 

Katz (1962) administered several tests from the Army Classification Bat- 
tery to 190 trainees In the Army's Automatic Data Processing Programming course 
He used these test results In an attempt to reduce the wasted training time and 
costs associated with the prevailing high attrition rate. Test scores were 
correlated with final grade In the course. A combination of verbal and arith- 
metic reasoning tests from the army battery yielded a correlation with grade 
of .61; the IBM Programmers Aptitude Test correlation with grade was .67; the 
multiple correlation of these tests with grade was .68. 
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Hollenbeck and McNamara (1965) rated the programming ability o£ 27 
programmer trainees at the end of a 26 week training program. These ratings 
were then correlated with scores on the Computer Usage Company Programmer Ap- 
titude Test, a computer-based apparatus test (rho - .30), and IBM's Data 
Processing Aptitude Test (rho 83 .49). Seiler (1965) obtained correlations 
ranging from .35 to .62 between supervisors' ratings of workers in electronic 
data processing jobs and combinations of three or more subtests from the GATB. 
Biamonte (1965) administered the Wonder lie Personnel Test plus three attitude 
tests)* ("authoritarianism," "conservatism," and "dogmatism") to 201 students In 
computer programming courses. Correlation of scores on the Wonder lie with 
course grade was .52, attitude scores had low negative correlations with grade, 
and the multiple correlation of all four tests with grade was .53. Thus in 
this study measures of attitudes contributed very little to the prediction of 
performance in a programming course. 

Currently, the literature reveals no test batteries designed expllclty 
for the purpose of identifying potentially good computer programmers. Most 
previous research has been confined to the use of a single test, usually of 
general Intelligence. The purpose of the present study was to perform some 
preliminary research for the development of such a battery, comparing the pre- 
dictive effectiveness of various existing tests singly and In combinations. 

A second purpose of this study was to examine the validity of two existing 
instruments for predicting success in programming: the new Computer Programmer 

key of the Strong Vocational Interest Blank (Campbell, 1966) and the Aptitude 
Test for Programmer Personnel (IBM, 1964). 

^California F -Scale of Authoritarianism, McClosky's Conservatism Scale, 
and Rokeach's Dogmatism Scale. 
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Method I 

The subjects were 68 students 58 men and 10 women, both undergraduate and j 

| 

graduate, enrolled In Computer Science 110 at Michigan State University* This | 

ten-week course Is an Introduction to elementary computer programming using | 

Fortran vocabulary* Final grade in the course was the criterion of programming j 

performance. This grade was based entirely on the total score amassed by the s 

student on 4 programs written as homework assignments, 3 hourly tests based j 

on lectures and assigned readings, and a 2-part final consisting of 1) questions J 

on theory and facts and 2) problems Involving the actual writing of programs* 

All test questions were constructed by the Instructor and were in objective 
form* 



Two predlctov variables were the IBM Aptitude Test for Programmer Person- | 



nel (ATPP)* and the Strong Vocational Interest Blank (SVIB) which were given 
to the students In two 1-hour sessions at the beginning of the course. Other 
predictors were scores on the College Qualification Test (CQT), which the 
students had taken upon admission to Michigan State University, and grade 
point averages (GPA’s) attained In all previous courses at Michigan State 
University* 

The ATPP Is a one hour paper and pencil test of reasoning ability de- 
signed by IBM to aid In the selection of computer programmers. It consists of 
3 subtests: Part I, a letter series test; Part II, a figure series test; and 

Part III, an arithmetical reasoning test* The manual reports a test-retest 
reliability from a sample of 144 college students of *88 for the total test 
(IBM, 1964). 



The authors express their appreciation to IBM for making these tests 
available* 
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The CQT is a general scholastic aptitude test for college applicants 
which yields a total and 3 part scores: verbal ability , numerical reasoning, 

and general information. Two scales from the SVIB were examined, the new 
Computer Programmer Scale and the Academic Achievement Scale. 

Results 

The simple correlations of the independent variables (ATPP total and 
Parts I, II, and III; CQT total and Verbal, Numerical, and Information; the 
Programmer and Academic Achievement Scales of the SVIB; and 6PA) with the 
dependent variable (final course grade) are reported in Table I below. 

Table I 

Simple Correlations of the 
Predictors with Course Grade 



GPA 


.68 


ATPP Part I 


.30 


CQT Total 


.49 


ATPP Part II 


.43 


CQT Verbal 


.37 


ATPP Part III 


.48 


CQT Numerical 


.53 


SVIB Programmer 


.33 


CQT Information 


.32 


SVIB Academic Ach. 


.33 


ATPP Total 


• 51 







All independent variables were significantly correlated with course 
grade at the .05 level. The best single predictor was GPA followed by the 
numerical subtest of the CQT, ATPP and CQT total scores, and Part III 
(numerical) of the ATPP. Table II contains the intercorrelations of the 
predictor variables. 







Table II 



Intercorrelations of Predictor Variables 



ATPP Total 


1 


— 




















ATPP Part I 


2 


.85 


— 


















ATPP Part II 


3 


.66 


.42 


— 
















ATPP Part III 


4 


.79 


.47 


.29 


-- 














CQT Verbal 


5 


.43 


.33 


.15 


.47 


— 












CQT Information 


6 


.45 


.36 


.20 


.45 


.56 


— 










CQT Numerical 


7 


.50 


.29 


.26 


.59 


.33 


.41 


— 








CQT Total 


8 


.57 


.41 


.24 


.61 


.88 


.81 


.64 


•• 






6PA 


9 


.44 


.23 


.15 


.57 


.50 


.50 


.51 


.62 


— 




SVIB Programmer 


10 


.02 


.02 


.05 


.00 


-.06 


.06 


.24 


.06 


.04 


— 


SVIB Aca. Ach. 


11 


.14 


.17 


.07 


.06 


.37 


.36 


.23 


.42 


.28 


.13 






1 


2 


3 


4 


5 


6 


7 


8 


9 


10 




i 



11 



A step-wise addition multiple regression analysis was computed using f 

I 



all Independent variables to determine the best combination 



for predicting 



course grade. The best two-score combination proved to be 6PA and Part II 
(Figure Series) of the ATPP (R 3 *76); the best three-score combination was 
obtained by adding the Computer Programmer Scale of the SVIB (R * .81). Further 
additions did not significantly improve the multiple correlation. 

Because 6PA accounted for such a large portion of the variance in course 
grade (r 3 .68) another multiple regression analysis was computed omitting 6PA 
as an independent variable. In this case the best two-score combination was 
CQT numerical and Part II of the ATPP (R 3 .61); addition of the Computer 
Programmer Scale of the SVIB raised the multiple correlation to .65; addition 
of CQT verbal gave an R of .68. Further additions did not significantly im- 
prove the multiple correlation. Table III summarizes the multiple regression 
analysis. 
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Table III 



Multiple Correlations o£ the 
Predictors with Course Grade 



R 



Bast combinations 



GPA and ATPP Part II 



.76 



GPA, ATPP Part II, and SVIB 
Computer Programmer 



.81 



Best combinations excluding GPA 

CQT Numerical and ATPP Part II 



CQT Numerical, ATPP Part II, and 
SVIB Computer Programmer 



CQT Numerical, ATPP Part II, SVIB 
Computer Programmer, and CQT Verbal 



.61 

.65 

.68 



Discussion 



Tests are probably used more o£ten In the selection of candidates for 
computer programmer training than in the hiring of computer programmers. Thus, 
despite Its limitations, the use in this study of final grade In a computer 
programming course as the criterion of programming success seems justified. 

In fact, the ATPP is designed specifically to select Individuals who will be 



successful In programmer training courses. 

The results suggest that the measures presently available do a pretty 
fair job of predicting grades. As is typical, the best single predictor of 
academic success was past academic success, l.e., GPA. Those wishing to 
select applicants for programmer training, however, will not always have grade 
point averages available. In this case, the results excluding GPA as an In- 
dependent variable would be of most interest. 

The ATPP was not appreciably better In predicting programming aptitude 
than the CQT, a general scholastic aptitude test. Indeed, with the exception 
of GPA, the highest single correlation with course grade was obtained with 
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the Numerical subtest of the CQT. Since generel scholastic test results are 

~fh£*W 

commonly available j|t would be more efficient to use these data rather than 
to replace it with ATPP data. However, examination of the multiple regression 
analysis suggests that subtest combinations can predict programming success 
better than any one total test. Numerical reasoning (CQT Numerical) and 
spatial reasoning (ATPP Part II— figure series) appear to be the most im- 
portant cognitive aspects for success in computer programming. 

Of special interest is the fact that, while the simple correlation of the 
SVIB Computer Programmer scale with grade was not particularly high (r * .33), 
this measure did significantly improve the prediction obtained from cognitive 
measures alone. As can be seen from Table II, the SVIB-CP scale was measuring 
something different from the other predictor variables. This unique "some- 
thing" (call it interest) does significantly help to predict computer science 
course grades. 
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